Papers with System-1-style Large Language Models
Teach Small Models to Reason by Curriculum Distillation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) show strong System-2-style reasoning, but at the cost of significant computational overhead. |
| Approach: | They propose a two-stage curriculum distillation framework which builds a robust internal problem-solving student model and then teaches the student model to externalize this knowledge as explicit reasoning. |
| Outcome: | The proposed model outperforms single-stage baselines on mathematical benchmarks and significantly outperformed LRMs on complex tasks. |